Allow keeping functional test datasets #3418
Conversation
PBENCH-1148 Allow disabling the `test_delete_all` test by setting the `KEEP_DATASETS` environment variable ... along with some squirrelly mechanism to get this set in the proper context. (Oddly, adding it to the `tox.ini` "safe" list wasn't enough to allow setting it outside; but we're already passing the server address as a command line parameter, so I added a second.)
dbutenhof
requested review from
ndokos,
MVarshini,
webbnh,
npalaska,
vishalvvr,
riya-17 and
siddardh-ra
|
Well this is interesting. Wonder if it'll recur ... ❔
|
Yeah, and, despite that, we managed to get a |
No, I think the failure was in the |
|
Mr. Jenkins managed to pull that container twice earlier in the job, but each time he pulls a fresh image. 😒 Google says that that error indicates a disconnect between Quay and its back end database. Looking at the |
|
And now it fails the legacy |
...and, this is by design, sort of. That is, we set We should probably change that to |
More efficient, to be sure; but in one sense that just makes the exposure points for whatever happened here less predictable. (I.e., whenever an executor finds a new pbench-ci-fedora image.)
Yeah, well it's particularly intriguing that it is revealed as a 500 error, which suggests that something triggered a quay "loophole". |
Actually, I don't think it diminishes the predictability, but, I think you're right that it doesn't necessarily improve it, either -- I think it has almost the same failure point. (That is, currently, we would fail any time we couldn't pull the image; using What we should do is pull the image at the start of the run (and fail early, if possible) and then use our cached image for the rest of the run. That might not be too hard to implement. (The problem is figuring out what "at the start of the run" means, across the cases of individual developer and CI, and from the
I think it was a transient on their end, especially since it hasn't recurred. |
Ah; so the failure in a critical network service was transient. Well that's OK then. 🙉 |
Not so much "OK" as "beyond my control" (and, I expect, beyond my ability to do very much about). But, if we didn't depend on pulling the container on every invocation, it might help.... 🥴 |
PBENCH-1148
Allow disabling the
test_delete_alltest by setting theKEEP_DATASETSenvironment variable ... along with some squirrelly mechanism to get this set in the proper context. (Oddly, adding it to thetox.ini"safe" list wasn't enough to allow setting it outside; but we're already passing the server address as a command line parameter, so I added a second.)Ultimately, this translates to a simple
jenkins/runlocal --keepcommand.